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Introduction 

The ground field k is algebraically closed and of characteristic zero. Let g be a reductive 
algebraic Lie algebra over Ik and a an involutory automorphism of g. Then g = g © g 1 is 
the direct sum of cx-eigenspaces. Here g is a reductive subalgebra and g x is a g -module. 
We also say that (g, g ) is a symmetric pair. Let G be the adjoint group of g and G the 
connected subgroup of G with LieG = go- The commuting variety associated with the 
involution a or symmetric pair (g, g ) is: 

£(fl0 = {(x,y) e 0i x 0! | [ar,2/] = 0}. 

The "usual" commuting variety is obtained as a special case of this construction. Indeed, 
let (gffig, g) be the symmetric pair such that g is embedded diagonally into g = g©g. Then 
gi ~ g as g-module and £(0i) is isomorphic to the commuting variety £(g) = {(x, y) e 
gxg | [x, y] = 0}. The latter was considered by Richardson in |13J. He proved that 
£(g) = G(txt), where t C g is a Cartan subalgebra, and therefore £(g) is irreducible. It is 
not yet known whether €(g) is normal or whether the ideal of <£(g) in k[g x g] is generated 
by quadrics. 

Let c C ji be a maximal abelian subspace consisting of semisimple elements. Any 
such subspace is called a Cartan subspace of 0i. All Cartan subspaces are G -conjugate, 
see 0. The rank of the symmetric pair (g, g ) is dim c, also denoted rk (g, g ). It is known 
that dimC(gi) = dimgi + dime and £ := G (cxc) is an irreducible component of <£(gi) 
of maximal dimension, see ITUl Sect. 3]. It follows that <£(gi) is irreducible if and only if 
£(0i) = U 

The irreducibility problem for C(gi) was first considered by Panyushev |10|. He no- 
ticed that C(gi) can be reducible 1 10 . 3.5]. On the other hand, in some particular cases 
€(0i ) is irreducible and enjoys a number of good properties. If (g, g ) is a symmetric pair 
of maximal rank (i.e., rk (g, g ) = rkg), then £(gi) is an irreducible normal complete inter- 
section and the ideal of £(gi) in k[gi xg x ] is generated by quadrics, see IflOl 3.2]. We refer 
to [12], [14], 1 15 1 for some recent results. 

The problem of irreducibility is closely related to some properties of rings of differential 
operators. Let V(g 1 ) denote the algebra of differential operators on g x with coefficients in 
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k[gi] and K, < X'(fli) the left ideal annihilating all G -invariant polynomials. The GVaction 
on gi allows us to interpret the elements of g as left-invariant vector fields on gi. This 
gives rise to a homomorphism of the universal enveloping algebra U(g ) to P(fii). If go 
contains no simple ideals of g, then this homomorphism is injective. We may therefore 
assume that U(g ) C £>(gi). It is easily seen that U(g ) C /C. Furthermore, it can be shown 
that if £(Qi) is irreducible and its ideal is generated by quadrics, then /C = £>(gi)U(g ). 
(See [7] for related results.) 

Suppose g is simple. Then the known results are 

• if rk (g, g ) = rkg (the maximal rank case), then <£(gi) is irreducible IfTOl ; 

• if rk(g,g ) = 1, then £(gi) is reducible unless (g,go) = (so m+ i,so m ) ITU : further- 
more, the number of irreducible components of £(gi) equals the number of non- 
zero nilpotent G -orbits in gi [12J; 

• for (s[ 2n ,sp 2n ) and (£ 6 , 5F4), the commuting variety £(gi) is irreducible |12|; 

• if (g, g ) = (so m+2 ,so 2 ©so m ), then £(gi) is irreducible C3. 

The goal of this article is to present some further results on the irreducibility problem. 
Let q x denote the centraliser of x G g. If x G gi, then a induces the symmetric decom- 
position q x = g , x © gi, x , where g 0):c := g n q x is the centraliser of x in g . If h G gi is 
semisimple, then (g^, Q 0ih ) is a symmetric pair, which is called a sub-symmetric pair asso- 
ciated with (g,go)- Let £(gi,h) denote the commuting variety associated with (Qh,9o,h)- 
Usually, the irreducibility of £(gi) is proved in the following way, which goes back essen- 
tially to Richardson |13|. First one has to show that <£(gi,/i) is irreducible for any non-zero 
semisimple h G gi; next, one has to verify that ({e}xg l e ) c €0 f° r arry nilpotent ele- 
ment e G gi. Actually, the verification in case of nilpotent elements readily reduces to 
a-distinguished elements (see Section [T] for precise definitions). To a great extent, the 
structure of <£(gi) depends on properties of a-distinguished elements. 

In Section [TJ we collect several useful facts concerning semisimple and nilpotent ele- 
ments in Qi. To implement the above program, we should be able to deal with the re- 
spective sub-symmetric pairs, and Proposition II .51 provides a description of all possible 
sub-symmetric pairs (g^, g ,/J in terms of the Satake diagram of (g, g ). 

In Section|2l we prove the irreducibility of the commuting variety for (so n+m , so„©so m ). 
This extends the result of IT5l , which refers to the case n = 2. The scheme of the proof 
is similar to that of |15|. But as it often happens, the argument in a general situation is 
shorter and simpler than in a particular case. Roughly speaking, the reason of success is 
that, for this symmetric pair, all a-distinguished elements are even. 

In Section|3j <£(gi) is shown to be irreducible for the symmetric pairs (£l 2ri ,£[ n ffis[ n ©ti) 
and (£ 6 ,5[ 6 ©s[ 2 ). Here we use the fact that in these two cases dimg! > dimg and the 
(closures of) subsets G ({e}xg l e ) cannot form an irreducible component of <£(gi). In the 
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classical case, £(di,h) is shown to be irreducible by induction, and in the £ 6 -case we rely 
on the explicit description of all sub-symmetric pairs (g^, Qo,h)- 

In II12L it was conjectured that <£(fji) is irreducible if rk (g,g ) > 1. This is, how- 
ever, false. In Section |4j we prove that £(gi) is reducible for the symmetric pairs 
(s[ n+m ,sl„©5[ m ©ti) with n 7^ m, (so 2 n,gU with odd n, and (£ 6 ,50i © ti). Their ranks 
are equal to min(n, m), [n/2], and 2, respectively Furthermore, we give am explicit lower 
bound on the number of the irreducible components of <£(gi) in the s[„ +m -case, which 
shows that this number can be arbitrarily large, see Proposition 14.41 In Section |5J we 
prove the reducibility of <£(gi) for (£ 7 ,50i 2 ©s[ 2 ) and (£ 8 , £7©s[ 2 ), using some ideas form 
the theory of principal nilpotent pairs. 

Taking into account the previously known results and the results of this paper, one sees 
that there remain only three cases (two classical series and one symmetric pair for £7), 
where the the answer is not known. In particular, for series A and 2, the irreducibility 
problem is completely solved. In Section|5J we discuss possible approaches to the remain- 
ing cases. 

Acknowledgements. This paper was written during our stay at the Max-Planck-Institut fur 
Mathematik (Bonn). We are grateful to this institution for the warm hospitality and support. 

1. Nilpotent elements, sub-symmetric pairs, and commuting varieties 

In this section, we deal with semisimple and nilpotent elements in Qi and their relations 
to the commuting variety. 

Let n be a non-negative integer. The set 



is locally closed. The irreducible components of the sets q\' are called the G -sheets of Q\. 

Lemma 1.1. Let § be a G -sheet of gi containing semisimple elements. Suppose that, for each 
semisimple h E S,we have ({h}xQi yh ) c € . Then {{x}xqi )X ) c € for each x e S. 

Proof. Let x E S. Since § contains semisimple elements, they form a dense open subset. 
Therefore, we can find a morphism 7 : k — > § such that 7(0) = x and 7(t) is semisimple 
for each t ^ 0. Then = lim^ 0i,7(t)/ where the limit is taken in an appropriate Grass- 
mannian. For each y E q 1jX , we can define elements y(t) E 0i, 7 (t) such that y = lim t ^ y(t). 
Since (x,y) = lim^ (7(£), y(t)) and (j(t),y(t)) E € for each t ^ 0, we conclude that 



0i 



(n) 



{£ E 0! I dim(G 



n} 



(x, y) E C - 



□ 



Lemma 1.2. |6. Prop. 5] For any x E 0i, we have dimg l a . — dimg 0i:r = dimg! — dimg . 
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Let ) denote the set of all nilpotent elements in g x . For any e G 91(gi), there is an 
s[ 2 -triple {e, /, h} such that / E gi and h G go |6|. Such a triple is said to be normal. Recall 
that e is called even if the eigenvalues of ad h on g are even. 

Definition 1. An element e G 91(gi) is said to be a -distinguished (in other notation, p- or 
(—1) -distinguished) if gi, e contains no semisimple elements of [g, g]. 

The following lemma appears in |15|, but the proof given here is shorter. 

Lemma 1.3. Suppose e G 01(gi) is even. Then e belongs to a G -sheet containing semisimple 
elements. 

Proof. Let (e, /, h) be a normal 5l 2 -triple. Since e is even, we have dimg^ = dimg e . Set 
e(t) := e — t 2 f for t G k. If t ^ 0, then e(t) is semisimple and conjugate to th. Therefore 
dimg e (i) = dimg h = dimg e and dimg ,e(t) = dimg 0)e by Lemma lL2l Clearly e(0) = e = 
lim t ^ e(t) and the G -sheet containing e contains also semisimple elements e(t). □ 

Let h G gi be a semisimple element. Recall that £(gi,h) denotes for the commuting 
variety associated with the sub-symmetric pair (g h , go,/i). 

Theorem 1.4. Suppose that each a -distinguished nilpotent element in g x is even and <£(gi.h) is 
irreducible for each semisimple non-zero h G g\. Then £(gi) is irreducible. 

Proof. Recall that £ = G (cxc) is an irreducible component of £(gi). Following the 
original proof of Richardson |13| (see also Ifl2l Sect. 2]), we show that £(qi) = <t . Let 

(x,y) G 

(1) Suppose there is a semisimple element h G gi such that [/i, x] = [h, y] = 0. This 
assumption is automatically satisfied if either x or y is semisimple. Moreover, if x (or y) 
is not nilpotent and x = x s + x n is the Jordan decomposition, then x s G gi and [x s , x] = 0, 
[x s ,y] = 0. 

Consider the sub-symmetric pair (gh,8o,h)- Replacing c with a conjugate Cartan sub- 
space, we may assume that h G c. Then c is a Cartan subspace of gi^. By the assumption, 
£(gi,/i) is irreducible. Therefore, (x, y) G G ,h(cxc) and hence (x,y) G Co- 

(2) Now we may assume that both x and y are nilpotent. Suppose first that there is a 
semisimple element h G gi such that [x, /i] = 0. Then (x, (1 — t)y + t/i) G C(gi) for each 
t G I and (1 — t)y + th is nilpotent only for a finite number of t's. Therefore, by part (1), 
one has (x, (1 — t)y + th) G (Co for almost all t. Since y = lim i ^ ((l — t)y + th), we get 
(x,y) G € - 

(3) It remains to handle the case in which both x and y are a-distinguished nilpotent el- 
ements. Then both x and y are even and by Lemma [131 x belongs to a G -sheet containing 
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semisimple elements. According to part (1), the assumptions of Lemma fl .11 are satisfied 
and it follows that (x, y) G <£ . □ 

In order to effectively apply Theorem 11.41 we need a description of possible sub- 
symmetric pairs. To this end, we recall the notion of the Satake diagram of a symmetric 
pair (g, g ), where g is semisimple. Roughly speaking, it is the Dynkin diagram of g, 
where each node is either black or white and some pairs of white nodes are connected by 
a (new) arrow. More precisely, let t be a cr-stable Cartan subalgebra of g containing c and 
let A be the root system of (g, t). Since t is cr-stable, a acts on A. It is possible to choose the 
set of positive roots, A + , such that if (3 G A + and (3(c) ^ 0, then a(/3) G - A + . Let II be the 
corresponding set of simple roots and a G II. Then the node corresponding to a is black if 
and only if a(c) = 0. Two different white nodes corresponding to an, ctj G II are connected 
by an arrow if and only if ai\ t = aj | c . The latter is equivalent to that a{oti) = —ctj- Looking 
at the Satake diagram, one immediately reads off many properties of a symmetric pair 
under consideration. For instance, 

r k (g, go) = (number of white nodes) — (number of arrows), 

and the centraliser of a generic element of c is the Levi subalgebra of g whose semisimple 
part is given by the subdiagram consisting of all black nodes. 

Let S be a Satake diagram. Then 5" is a subdiagram of S if 5" is obtained from S by re- 
peating the following procedure. Namely, one can remove one white node, if it is not 
connected by an arrow; or one can remove a pair of nodes connected by an arrow. 
Let h G fli be semisimple. Set p = [$h, 0a] and pi = g» fl p. Then (p, p ) is said to be a 
reduced sub-symmetric pair of (g,go)- Clearly, <£(0i,/i) is irreducible if and only if £(pi) is 
irreducible. So, it is enough to describe all reduced sub-symmetric pairs. 

Proposition 1.5. A symmetric pair (p, p ) is a reduced sub-symmetric pair of (g, O ) if and only 
if S p , the Satake diagram of (p, p ), is a sub-diagram of S s , the Satake diagram of (0, O ). 

Proof. Without loss of generality, we may assume that h G c. Then a(h) — if the node 
corresponding to a G II is black. Since we are only interested in possible types of cen- 
tralisers, we may assume that h lies in the Q-form cq. Here cq is the Q-span of all restricted 
roots f3\ c , (3 G A, and c is identified with c* using the restriction of the Killing form. More- 
over, using the action of the little Weyl group on Cq, we may assume that a(h) > if the 
node corresponding to a G II is white. Then the Dynkin diagram D p of p is a subdiagram 
of D s consisting of all a G II such that a(h) = 0. In particular, D p contains all black nodes 
oiS B . 

Set c p := c n p. It is a Cartan subspace of pi and c = c p © 31, where 31 lies in the centre of 
Q h . If ctj is a simple root of p, then a* (31) = and ct^Cp) = if and only if «j(c) = 0. Hence 
the nodes do not change their colours. Suppose a { and a,j are connected by an arrow in 
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S g . Then (ccj — <x,-)(c) = 0. In particular, ai(h) = if and only if ctj(h) = 0. If a; and ctj 
belong to .Dp, then they are still connected by an arrow, since c p C c. Conversely, if otj 
are simple roots of p and they are connected by an arrow of S p , then (a* — otj)(tp) = and 
(ofj — aj)(3i) = 0. Therefore (c^ — <x,)(c) = 0, and a if otj are connected by an arrow in S s . 
Thus, the Satake diagram of any reduced sub-symmetric pair is a subdiagram of S g . 

Conversely, let S' be a subdiagram of S s in the sense of the above definition. Consider 
the subspace c' C c, where all simple roots corresponding to the nodes from S g \ S' van- 
ish. Then, for a generic h E c', we obtain the reduced sub-symmetric pair whose Satake 
diagram is S'. □ 



2. IS IRREDUCIBLE FOR (fl, fl ) = (S0 n+m , S0 n ©S0 m ) 

In this section G = SO n+m , g = so n+m , and g = so n ffiso m . Let V = k n+m be a vector 
space of the defining representation of g. Then we have a Go-invariant decomposition 
V = V a ®V b , where K = k n , V b = k m and gi = k n ®k m as a L7 -module. Let ( , ) denote the 
non-degenerate symmetric G-invariant bilinear form on V. In order to apply Theorem ll.4l 
we need a description of semisimple and nilpotent elements in g 1 . 

Lemma 2.1. Let h e Qi be a semisimple element. Then the symmetric pair (Qh,Qo,h) is a direct 
sum (0- =1 (g^,so fc J) © (so n+m _2fc,so n _ fc ©so m _ fc ), where k = Y Ji h- 

Proof. Let v x EVbe an eigenvector of h such that h-v x = Xv x and A ^ 0. Since /i preserves 
the symmetric form ( , ), i.e., (h-v x ,v x ) + {v x ,h-v x ) = 0, we have (fA,^) = 0. Also, if 
h-v = Xv, h-w = jiw, then (i>, w) = unless A = — //. Let {±Aj, | i = 1, . . . , r} be the set of 
the eigenvalues of h. Then there is an orthogonal /i-invariant decomposition 

V = (V Xl ®V- Xl ) © • • • © (V Xr ®V- Xr ) © v . 

Here each is an isotropic subspace, (V^, V Xj ) = unless Aj + Xj = 0, and (V , V±\ i ) = 
for each Aj. Therefore g h C (® r i= iSo(V Xi (BV^ Xi )) © so(Vo). More precisely, if dimV^ = h 
and k = J2i h, then 9h = (0< 0^) © so„ +m _ 2 /c- 

Now it remains to describe g 0) ^ = We may assume that a is a conjugation by a 

diagonal matrix A e O n+m such that A\ Va = —id and A\ Vb = id. Since a(h) = —h, we 
have A ■ V Xi — V- Xi and A ■ V = V . Moreover, A determines a non-degenerate symmetric 
form ( , )a on each Va 4 by the formula (i>, = (i> , A ■ u>). Therefore, each so(V Xi ®V- Xi ) 
is cr-invariant, (so(VA i ©V r _A i )) CT — so^Qso^, and = so^. Finally, the restriction A\ Vo 

has signature (n — k,m — k). Thus (50„ +m _ 2 A : ) fJ = so n ^ k @50 m - k . □ 

Note that this result can also be deduced from Proposition II .51 
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Recall several standard facts concerning nilpotent elements in gl(V). Suppose e G gl(V) 
is nilpotent and m = dimKer (e). By the theory of Jordan normal form, there are vec- 
tors wi,...,w m G V and non-negative integers di,...,d m such that e di -Wi = and 
{e s -Wi | 1 < i < m, < s < di} is a basis for V. Let V[i] C V be the linear span of 
{wi, e-Wi, . . . , e* -1 -^}. The spaces {V[i]} are called the Jordan (or cyclic) spaces of e, and 
V = ®T=iV[i\. 

Lemma 2.2. Suppose e G 9t(0i). Then the cyclic vectors {m^}™! and hence the cyclic spaces 
{V[i\}'s can be chosen such that the following properties are satisfied: 

(i) there is an involution in i* on the set {1, . . . , m} such that: di = di*; i = i* if and only 
if dim V[i] is odd; (V[i\,V[j}) = 0ifi^ j*; 

(ii) a(wi) = ±Wi. 

Proof. Part (i) is a standard property of the nilpotent orbits in so(V), see, for example, (TJ 
Sect. 5.1] or |5, Sect. 1]. Then part (ii) says that in the presence of the involution a cyclic 
vectors for e G 9T(gi) can be chosen to be cr-eigenvectors, see 00 Prop. 2]. □ 

For each e G ^1{qi), we choose cyclic vectors {wi} as prescribed by Lemma l2~2l Let us 
say that e s -Wi is of type a if a(e s -Wi) = —e s -Wi, i.e., e s -Wi G V a ; and e s -Wi is of type b if 
e s -Wi G V b . Since cr(e) = — e, if e s -Wi G V a , then e s+1 -Wi G V b and vice versa. Therefore each 
string (wi, e-Wi, . . . , e rfi_1 -Wj) has one of the following types: 
aba . . .ab, bob . . . ba, aba . . . ba, bob . . .ab. 

Proposition 2.3. Every a -distinguished element e G 9t(fli) is even. 

Proof. Suppose e G 9T(gi) is a-distinguished and the cyclic spaces {V[i]} are chosen as 
prescribed by Lemma l2~2l Assume that dim V[i] is even for some i. By 1 8 , Prop. 2], if V[i\ is 
of type aba . . .ab, then V[i*\ is necessarily of type bob . . .ba, and vice versa; i.e., if cr(wi) = 
TWi, then a(wi*) = ±Wj*. It is easily seen that so(l / [z]©l / [z*]) n g e contains a cr-stable 
subalgebra I that is isomorphic to sl(ku>j © kwi*) = sl 2 . More precisely, any unimodular 
transformation of k.Wi®k.Wi* can be uniquely extended to an element of so(V[i]©V[i*]) f)Q e . 
Since the restriction of a to I is non-trivial, h = I H gi C g e contains semisimple elements. 
This means that such e is not a-distinguished. Hence, all V[i] are odd-dimensional. Since 
a nilpotent element of so at is even if and only if all the numbers dim V [i] have the same 
parity, we are done. □ 

Theorem 2.4. For (g,go) = (so„ +m ,so n ©so m ), the commuting variety C(fli) is irreducible. 

Proof. Let c C gi be a Cartan subspace. We argue by induction on dim c. The base of 
induction is the rank 1 case (so n+1 ,so n ), where the irreducibility of <£(gi) is proved in |12J, 
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Let h be a semisimple element of gi. By Lemma IZTl (g^, g jh ) = (©^(glfc.jSOfcJ) © 
(so n+m -2k, so n -k®so m -.k)- Each pair (g[ fc .,s0fcj is a symmetric pair of maximal rank, hence, 
the corresponding commuting variety is irreducible, see fTUl (3.5)(1)]. Clearly, the com- 
muting variety corresponding to a direct sum of symmetric pairs is the product of the 
commuting varieties corresponding to the summands. Therefore, using the inductive hy- 
pothesis, we conclude that is irreducible. Taking into account Proposition 12.31 we 
see that the assumptions of Theorem II .41 are satisfied and C(fli) is irreducible. □ 

3. IRREDUCIBILITY OF <£(fli) FOR SYMMETRIC PAIRS WITH dimgi > dimg 

In this section we show that the method of proving irreducibility of <£(fji) in the 
maximal rank case 111 OH actually applies in a more general setting of symmetric pairs 
with dimgi > dimg . Practically, this yields the irreducibility for two new pairs: 

(sl 2 n,sl„©s[„©ti) and (£ 6 ,s[ 6 ©sl 2 ). 

Recall that (g, g ) is of maximal rank if and only if the Satake diagram has no new arrows 
and all the nodes are white. 

Lemma 3.1. Suppose all nodes of the Satake diagram of (g, g ) are white and there is at least one 
that is not connected by an arrow with any of the others. Then dim gi > dim g . 

Proof. Choose a semisimple element h e gi such that gi^ = c is a Cartan subspace. Then 
(do)h coincides with the centraliser of c in g . Therefore the structure of Q ,h can be read off 
from the Satake diagram. In our case g 0th is a torus and dimg ,h = rkg — dim c < (rkg)/2. 
Hence dimg 0i h < dim c, and dimg < dimg! in view of Lemma H~2l □ 

Another simple observation is that gixg x = T*(gi) and the commutator morphism 
■0 : 0i x 0i ^ So is the moment map for the natural Hamiltonian action of G on T*(qi). 
Therefore tp is dominant if and only if the generic stabiliser for the diagonal action of G 
on Qi XQi is finite. This is always true if the centraliser of c in g is a torus, i.e., the Satake 
diagram contains no black nodes. 

Lemma 3.2. Suppose dimg! > dimg , ip is dominant, and £(gi,/J is irreducible for all non-zero 
semisimple h e g^ Then C(fli) is irreducible. 

Proof. The dominance of ip implies that if ^ 0, then all irreducible components of 

are of dimension ^ 2 dimgi — dimg . Since 2 dimg i — dim g > dimgi, all irreducible 
components of V'^ 1 (0) = <£(gi) are of dimension > dimgi. 

Let x E Qi. Then G ({x}, Qi >x ) is an irreducible subvariety of <£(jji) of dimension dimgi. 
If x is not nilpotent, then G ({x}, Q ljX ) C € , see proof of Theorem 11.41 part (1). As is 
well known, G has finitely many nilpotent orbits on g lt Hence C(fli) is a union of £ and 
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finite many irreducible varieties of dimension dimg!. Since each irreducible component 
of €(qi) has dimension greater than dim g lr <£(gi) must be irreducible. □ 

Theorem 3.3. The commuting varieties associated with (st 2 „, sl n (Bsl n (Bti) and (£ 6 , st 6 ffist 2 ) are 
irreducible. 



Proof. First consider the classical case. We argue by induction on n. If n = 1, then (g, g ) = 
(303,302) and the corresponding commuting variety is irreducible. In order to make an 
induction step, we use Lemma 13.21 Here dimgi = 2n 2 , dimg = 2n 2 — 1. The generic 
stabiliser for the action of G on g x is an (n— l)-dimensional torus. It remains to show that 
£(f?i,ft) is irreducible for all non-zero semisimple h G gi. 



Let h G gi be semisimple. Using either Lemma 11.51 or the same argument as in 
Lemma 12.11 one can show that the semisimple part of (Qh,Qa,h) is a direct sum of 

(B r i=1 {slki®slk i ,slk i ) and (s[ 2 n-2A : ,s[ n _fc©s[ n _ fc ©ti). Therefore £(Qi,h) is irreducible and we 
are done. 

Consider now the exceptional case. The Satake diagram of (8-6, sl 6 © sfe) is: 

(_ i > ) 

o — o — o — o — o 

o 

It contains no black nodes, hence ip is dominant. There are two nodes which are not con- 
nected by an arrow with others, hence, by Lemma l3~Tl dim gi > dim g . In fact, dim gi = 40 
and dim g = 38. To apply Lemma l3~2l one has to find all Satake subdiagrams and verify 
that the associated commuting varieties are irreducible. 

If a Satake diagram is disconnected as a graph, then the corresponding symmetric pair 
is a direct sum of two non-trivial symmetric pairs. Since the commuting variety associated 
with a direct sum of symmetric pairs is a product of the commuting varieties associated 
with the summands, we can restrict ourselves to connected subdiagrams. There are seven 
of them. 



o — o — O — O — O O — O — O O — O O — O O — O — O O OOO 



o 



A 



These diagrams correspond to the following symmetric pairs: 

(s[ 6 ,s[ 3 ffis[ 3 ©ti), (so 8 ,so 5 ©so 3 ), (s[ 3 ©s[ 3 ,s[ 3 ), 

(sl 4 , sl 2 ®sh®ii) = (so 6 ,so 4 ©so 2 ), (sl 2 ffls[2,s[ 2 ), (s[ 3 ,so 3 ), (sl 2 ,50 2 ). 

For all of them the irreducibility of £(gi) is already established. □ 
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4. Short Z-gradings and reducible commuting varieties 

Let g = g(— 1) © g(0) © g(l) be a short Z-grading of a reductive Lie algebra g. Set 
go := g(0). Then (g, g ) is a symmetric pair with gi = g(— l)©g(l). Let c C gi be a Cartan 
subspace and let c(±l) denote the image of c under the projection to g(±l). As c consists 
of semisimple elements, we have c fl g(±l) = and dim c = dim c(l) = dim c(— 1). Below, 
we consider the diagonal C7 -action on g(l) x g(l). 

Lemma 4.1. Suppose £(gi) is irreducible. Then 



(*i) C7 (c(l)xc(l))=g(l)xg(l). 

Proof. Notice that g(l) is a commutative subalgebra, hence g(l) x g(l) c C(jji). Since 
c C c(— 1) © c(l), we have 



G (cxc) c C7 (c(-l)xc(l)xc(-l)xc(l)) c G (c(-l)xc(-l)) x G (c(l)xc(l)). 



Hence G (cxc) n (g(l)xg(l)) c C7 (c(l)xc(l)). If <£(gi) is irreducible, then 

C7 (cxc) n (g(l)xg(l)) = g(l) x g(l), and we get equality □ 

Similarly, one can consider condition (4-i) with c(— 1) in place of c(l), etc. It is easily 
seen that (4i) is satisfied if and only if (4 i) is. For this reason, this common condition is 
denoted by (4)- 

Corollary 4.2. If condition (4) is not satisfied, then <£(gi) has at least three irreducible compo- 
nents. 

Proof. Along with the standard component € , we have at least two other components 
determined by g(l)xg(l) and g(-l)xg(-l). □ 

Now we give three examples of symmetric pairs arising from short Z-gradings such that 
condition is not satisfied. 

Example 4.3. (g, g ) = (sl n+m , sl n © si m © ti). 

It will be easier to work with the symmetric pair (gl n+m , gt n © gl m ) having the same com- 
muting variety. We use the following matrix model. Let V be an (n + m) -dimensional 
vector space and g = gl(V). Let V = K © 14 be a vector space decomposition with 
dim\4 = n and dimVf, = m. Then g(l) = Hom(V a , H)/ g(— 1) = Hom(Vb, V a ), and 
g(0) = Hom(\/ a , V a ) © Hom(V r 6 , V b ) ~ gl„ © gl m . The corresponding involution a is the inner 
automorphism determined by the matrix A E GL(V) such that A\ Va = —id, A\ Vb = id. 

In the matrix form, every £ G gi is depicted as follows: £ = ^ ^' w ^ ere 

£(1) G Hom(\4, Vt) is an m x n matrix and £(—1) G Hom(Vb, V a ) is an n x m matrix. 
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Assume that n < m. To each pair (£, 77) G g x x g lr we assign the mx2n matrix -D(£, 77) := 
(£(1) I ?7(1))- We will show that rk£)(£, r/) ^ n for any (£, 77) G G (c x c), whereas there are 
some (£,77) G 0(1) x 0(1) such that rk 77) > n. This means that condition (4) cannot 
be satisfied here. 

Let c(— 1) be the set ofnxm matrices 

\ f x x ... 0... 0\ 

x 2 ... : ... : 







1 ., 







Xi, 



x n G 



> . 



0/ 



Then c := 



^ ^ I |Me c(— 1) ^ is a Cartan subspace of 0i. It is clear from the 
definition that rk -D(£, 77) ^ n for any (£, 7/) G cx c. Next, an easy verification shows that 

'b 



this remains true for the elements of G (c x c). Indeed, let g 

^(-l)^- 1 



c 



G G . Then 



Ad(sK 



where i3 



c^i)^- 1 







. It follows that D(Ad(g)-£,Ad(g) -77) = CD(£, rj)B, 



B 



-1 







_j . Hence rkL>(£, 77) = rkL>(Ad(g)-£, Ad(g)-ri) for any p G G . 



73- 

On the other hand, it is clear that there are two m x n matrices £(1),77(1), i.e., two 
elements of 0(1), such that rk (£(1) | r?( 1) ) = min(2n, m) > n, since m > n. This completes 
our argument. 

Thus, £(01) has at least three irreducible components, by virtue of Corollary 14.21 However, 
this lower bound is far from being precise. Surprisingly, C(0i) can have arbitrary many 
irreducible components. 

Proposition 4.4. Suppose (0,00) = (gt n +m> d^n®d^m) w ^ n ^ m. Then <£(jji) has at least 
2 min(277,, m) — 2n + 1 irreducible components. 

Proof. For £ = ^ G 01 and 77 = ( ° ^ G 01, we set Di(£, 77) := (X | Z) (as above) 
and -D 2 (£, 77) := (F* \ U l ). These are two m x 2n-matrices. 

1. First, we are going to prove that rk73 1 (£,77) + vkD 2 (^,r]) < 2n if [£,77] = 0. Suppose 
rkX = r. Replacing (£,77) with (Ad(<7)-£, Ad(g)-T]) for a suitable # G Gq, we may as- 



sume that X 



(xij) such that 



1 if i 



r and x« = for all other Then 



rk Di (£, 77) = r + rk z^, where Z\ is the (m — r)xn matrix consisting of the last (in — r) rows 
of Z. Condition [£, 77] = means that Xf/ = zTF and YZ = UX. The last r rows of X and 
hence of XU are zero. Therefore Z{Y = and rk F ^ n — rk Z\. 



12 



D. PANYUSHEV AND O. YAKIMOVA 



Let X , Z , U be the sub-matrices of X, Z, and U, respectively, consisting of the first 
r rows. From the equality XU = ZY and our assumption on X, we then obtain Z Y = 
XqU = Uq. Therefore the rank of the mx(n + r) matrix (V* | Uq) is not grater than rk Y and 
hence rk (Y l | [/*) ^ rk Y + (n — r). Thus, 

TkDi(C,,i]) + rk_D 2 (£,?7) ^ (r + rkZi) + (n — rkZi + n — r) = 2n. 

2. Let P q be the closed subset of <£(gi) defined by the conditions: 

rk Di(£,r]) ^ q and rk D 2 (£, 77) ^ 2n — g. 

As was just proved, <£(0i) = U-P-r Since 77) is an mx2n-matrix, rk_Dx(^,?7) ^ 

min(2n, m). Hence P q C P m - m (2n,m) if q > min(2n, m). Similarly, for q < 2n — min(2n, m), 
we have P g C P r , where r = 2n — min(2n, m). This implies that 

min(2n,m) 

cui) = U p * ■ 

2n— min(2n,m) 

Let us show that P q <£. IJ P r for each q E [2n — min(2n, m), min(2n, m)\. Since C(fli) is 
invariant under the transpose, there is no harm in assuming that q ^ n. 

Keep the previous notation. Take (£, 77) e C(fli) such that the submatrices X , Z , U 0/ 
etc. satisfy the following conditions: rkX = rkX = n (i.e., r = n),U = 0, rkZi = q — n, 
and Zq = 0. Let Y and Y"i denote the submatrices of Y consisting of the first n columns 
and the last m — n columns, respectively. Then the condition [£,77] = is equivalent to 
that Z{Y = and Y±Zi = 0. One can choose Y satisfying these conditions and such that 
Yi = and rkY = n — xkZ\. Then rk Z?i(£, 77) = q and rk D 2 (£, 77) = 2n — q. That is, we 
found a point in Pq \ |J P r . It follows that each P q contains an irreducible component of 

<£($ji) that does not belong to the other P r , and we are done. □ 

Remark. Motivated by the formula of Proposition 14.41 we set F(n,m) = 2 miri(2n, m) — 
277, + 1 if 1 ^ n ^ m. For n = 1 and m ^ 2, we have rk (gt m+1 , gl m © g^) = 1, and it was 
shown in 11121 that <£(gi) has three irreducible components. Since F(l,m) = 3 if m ^ 2, 
Proposition 14.41 gives here the exact value for the number of irreducible components. For 
m = n, we also obtain the exact number of irreducible components, in view of Theo- 
rem 13.31 It is curious to observe that F(n, m) = F(m — n, m). Hence P(m — 1, m) = 3, and 
one may ask whether it is true that, for the symmetric pair (gl 2 m-i> fl^m © S^m-i)' the va- 
riety <£(gi) has exactly three irreducible components. More generally, one may conjecture 
that the varieties {P q } in the previous proof are irreducible and therefore the number of 
irreducible components is always equal to F(n, m). The equality F(n, m) = F{m — n, m) 
also suggests that there might be a natural bijection between the irreducible components 
of the commuting varieties associated with (gl m+n , gl m © gl„) and (g[ 2m _„, 0t m © gL-rJ- 
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Example 4.5. (g,g ) = (so 2 „,0lj. 

Let V be a 2n-dimensional vector space and g = so(V) = so 2n . Consider a decomposition 
of V into a direct sum of two isotropic subspaces V = V + © V— Let A G 0(V) be such that 
A\ v+ = id and A\y_ = —id. Then the conjugation by A defines an involution of such that 
0o = $n — $KV+) — &K V -) and 0i c Hom(y + , V-) © Hom(y_, V+). More precisely, one can 
choose bases for V+ and V- such that 

s g GL n 1 , fli = |fy *1 l x > y efii n ,A: = -A; t ,y = -y* 

0i n Hom(V r + , V-) and g( — 1) = 0i fl Hom(V_, V+). For a pair (£,77) G gi x g x 
with £ = I ^ and 77 = ^ ^J, we set Di(£, 77) := (X | Z). We take the Cartan 

subspace c C gi consisting of skew-symmetric anti-diagonal matrices, i.e., 

X = -X* and X = (xij), Xij = if z + j ^ n + 1 

Now assume that n = 2k + 1. Then rkZ?i(t, /i) < n for any (i, ft,) 

'b 

(5 





S=L , ] GG and^= A ] G 01, then Ad(#K 





Therefore, Di(Ad(<7)-£, Ad (g) -77) = BDi(£, rf)B, where S = ( ( ) is a non-degenerate 



5* 
£ f 

2nx2n matrix. Hence rkD 1 (Ad(g)-£ i , Ad(g) -rj) = rk.Di(£, 77) and rk L>i (t, h) < n for each 
pair (t, h) G <£q. 

Let X, Z G 0(1) be skew-symmetric nxn matrices of rank 2k such that the last column 
and the last row of X are zero, and the first row and the first column of Z are zero. 
Clearly, if jfc ^ 1, then rk (X\Z) = 2k + 1 = n and therefore (X, Y) G" G (c(l)xc(l)). Thus, 
for (so 2n , g[ n ) with odd n ^ 3, condition (4) is not satisfied. 

Example 4.6. Consider now the symmetric pair (£ 6 , soi © t x ). 

Here g(l) and g(— 1) are different half-spinor representations of soio- Let t be a Cartan 
subalgebra of so w and {e\, . . . , e 5 } an orthonormal basis of Iq. Let {vri, . . . , 7r 5 } be the 
fundamental weights of so w such that 

7r 4 = (si + £ 2 + e 3 + e 4 - e 5 )/2 and tt 5 = (ej + e 2 + £3 + £4 + e 5 )/2 

are the highest weights of half-spinor representations. (See |9, Reference Chapter] for 
more details on the notation.) Let 71(f) denote the simple S0i -module with highest 
weight ip. We assume that g(l) = 1Z(ir 5 ) and g(— 1) = 7£(7r 4 ). The rank of (£ 6 ,soi © ti) 
equals 2 and a Cartan subspace c C gi can be chosen such that c(l) is a t-stable subspace 
of g(l) with weights 7r 5 and (ei — e 2 — e 3 — e 4 — e 5 )/2. 
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Because the weights of c(l) belong to an open half space of tQ, c(l) x c(l) lies in the null- 
cone of the Spin 10 -module g(l) x g(l). (Actually, similar assertion holds for any number 
of copies of c(l) and g(l). Therefore if / G k[g(l)xg(l)] Spmi « is a homogeneous polyno- 
mial of positive degree, then it vanishes on G -(c(l)xc(l)). Hence the very existence of 
non-trivial Spin 10 -invariant polynomials on g(l) x g(l) will imply that (4) is not satisfied. 

As S 2 (TZ(Tti)) = 71(2^4) © TZ(tti), there are the following Spin 10 -equivariant inclusions: 

5 4 (^(7r 4 )©^(7r 4 )) D S 2 {n{7r A )) ® <S 2 (ft(7r 4 )) D Ufa) ® Ufa) . 

Since (1Z(iTi)(g>1Z(iri)) Spmi0 7^ 0, we obtain a desired non- trivial Spin 10 -invariant of degree 
4ink[g(l)xg(l)] = S'(n(n 4 )®TZ(n 4 )). 

5. Principal nilpotent pairs and commuting varieties 

In this section, we prove that <£(gi) is reducible for two symmetric pairs related to excep- 
tional Lie algebras. 

Let s be the centraliser in g of a generic element of c. Since c is diagonalisable, g x (~)g y D s 
for any x, y G c. Therefore, dim(g x D g y ) ^ dims for any pair (x,y) G C7 (c x c) = Co- 
in what follows, we write g x :V for g x n g y . Assume that we managed to find a pair of 
commuting elements £, 77 in g x such that dim g 5 r) < dims. Then clearly (£, 77) g £ , and we 
conclude that £(gi) is reducible. 

Recall that dim g^ ^ rk g for any pair of commuting elements. This follows from the 
irreducibility of the "usual" commuting variety <£(g). Therefore, one may try to find a 
symmetric pair such that dims > rkg, whereas dimg^ = rkg for some (£,77) G <£(gi). 
This will be done with the help of theory of principal nilpotent pairs. 

Definition 2 (Ginzburg [4J). A pair (e^ e 2 ) G g x g is said to be principal nilpotent (pn-pair, 
for short) if 

[ei, e 2 ] = 0, dimg eiie2 = rkg, 

and there exists a pair of semisimple elements (Tti, h 2 ) G g x g such that ad hi and ad h 2 
have integral eigenvalues and [hi, h 2 ] = 0, [h i: e 3 ] = dijej G {1, 2}). 

Since the eigenvalues of ad hi and ad h 2 are integral, we can consider the corresponding 
Z x Z-grading of g: 

= © fl(«, j), where g(z,j) = {a; G g | [/i x ,a;] = ix, [h 2 ,x] = jx}. 

Notice that e\ G g(l,0) and e 2 G g(0, 1). Using this bi-grading, we define a symmetric 
pair (Z 2 -grading) by letting g = ^j-^ From this construction, it follows that 

i+j is even 

(ei, e 2 ) G £(gi). If we are lucky and the resulting symmetric pair has the property that 
dims > rkg, i.e., s is not Abelian, then we certainly know that €(gi) is reducible. 
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It was shown by Ginzburg Q that the number of G-orbits of pn-pairs is finite, and a 
complete classification of such pairs in exceptional Lie algebras is obtained by Elashvili 
and Panyushev [4, Appendix]. Using that classification, it is not hard to find out which 
pn-pairs lead to symmetric pairs (as described above) such that dims > rkg. Actually, 
there is a unique pn-pair with this property if g is exceptional. This pair occurs in the next 
theorem. 

Theorem 5.1. For the symmetric pairs (£ 7 ,soi 2 ©s[ 2 ) and (£ 8 , £ 7 ©s[ 2 ), the variety £(gi) is re- 
ducible. 

Proof. 1. For the simple Lie algebra £ 7 , there is apn-pair (ei, e 2 ) such that both e\, e 2 belong 
to the .EV-orbit of type A 4 + A\. (See below an explicit construction of such a pair.) The bi- 
grading corresponding to this pair is depicted in 111! Fig. 2]. This shows that dimg = 69 
for the corresponding Z 2 -grading. Hence g — S0i 2 ffis[ 2 . For this symmetric pair, we have 
[s,s] ~ 3A±. Hence (ei,e 2 ) <£ - 

2. The same pn-pair can be used for proving reducibility in the £ 8 -case. Using the stan- 
dard Lie algebra embedding g := £ 7 c £s =: g, the first symmetric pair can be re- 
garded as a subpair of the second. This simply means that we have two embeddings 
0o = soi 2 ©5l 2 C £ 7 ffis[ 2 = 0o and 0i C 0i. Then (e 1 ,e 2 ) e £(gi) C C(fli). As above, we 
wish to prove that (ei, e 2 ) does not belong to the irreducible component <£ — G (c x c). 
The two symmetric pairs in question have the same rank (namely, 4), so we actually may 
assume that c = c. For a generic element of c, its stabiliser in has the semisimple part 
of type D A . Therefore dims = 28 + 4 = 32. On the other hand, we will prove that 
dim0 eiie2 = 26 < 32, which establishes the desired reducibility. 

To this end, we need an explicit construction of the pn-pair (ei, e 2 ) in £ 7 . According 
to classical results of E.B. Dynkin [3], £ 7 contains a unique (up to conjugation) maximal 
semisimple subalgebra of type A 2 . The embedding A 2 C £ 7 has many remarkable prop- 
erties. For instance, E 7 /A 2 is an isotropy irreducible homogeneous space. Here £ 7 /.A 2 is 
the simple yi 2 -module TZ(Aty 1 +Att 2 ). Let a, (3, a + (3 be the positive roots of A 2 . There are 
two non-equivalent choices of nontrivial pn-pairs in A 2 . One takes (ei, e 2 ) = (e a , e a+ p) or 
(ep, e a+ p). Then one easily verifies that these two pairs remain principal also in £ 7 . Fur- 
thermore, one can prove that these two pairs are conjugate with respect to the group E 7 , 
so one obtains a single conjugacy class of pn-pairs in £ 7 . 

In order to compute dim0 eiie2 , we first decompose = £ 8 as £ 7 -module, and then 
further as >l 2 -module. First we notice that 

0k = 0©2V©3I, 

where V is the 56-dimensional simple £ 7 -module and I is the trivial one-dimensional 
module. Since (ei, e 2 ) is a pn-pair in £ 7 , we have dim g ei , e2 = 7. To compute the fixed-point 
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subspace of (ei, e 2 ) in V, we use fact that V\a 2 — 7^(67Ti) ©7^(67r 2 ), see Table 24]. Finally, 
one easily computes using the above two presentations of (ei, e 2 ) that dim7?.(67ri) ei ' e2 = 1 
and dim7^(67r 2 ) ei ' e2 = 7 (or vice versa). Altogether, we obtain 

dim eii62 = 7 + 2(7 + 1) + 3 = 26. 

This completes the proof of reducibility in the £ 8 -case. □ 

6. A FEW REMARKS ON THE REMAINING SYMMETRIC PAIRS 

There are three cases for which the irreducibility of is not known yet. They are listed 
below, together with their Satake diagrams. 

1) (sp 2n+2m , sp 2n ©sp 2m ), n ^ m ^ 2 

2) (so 4n ,gi 2n ),n > 2 

3) (£ 7 ,£ 6 eti) 




Items 2) and 3) are symmetric pairs of Hermitian type, i.e., g is not semisimple. Such 
symmetric pairs (and the corresponding symmetric spaces) can be either of tube or non- 
tube type. All symmetric pairs of Hermitian type arise form short Z-gradings of g. This 
construction is explained in Section HI There are many characterisations (definitions) of 
symmetric pairs of tube type. One of the possible definitions is the following. Let G' be 
the commutator group of G . Then (g, g ) is said to be of tube type if k[g(l)] G o ^ k. Then 
one can prove that (g, g ) is of tube type if and only if the dense (nilpotent) orbit in G-g(l) 
is even if and only if the Cg-action on g(l) is stable. Yet another characterisation is that 
the symmetric pairs of tube type are those related to simple Jordan algebras. 

Examples l4.3l (with n ^ m). 14.51 (with n odd), and 14. 6l represent all Hermitian symmetric 
pairs of non-tube type, and we have proved that for all these cases <£(gi) is reducible. On 
the other hand, for the symmetric pair (st 2n , sl n © sl n © ti), which is of tube type, <£(gi) is 
irreducible, as proved in Section |3l Other symmetric pairs of tube type are (sp 2n , gl n ) and 
(so n+2 ,so n © so 2 ), where the irreducibility is also known. This suggests that for items 2) 
and 3), which are of tube type, £(gi) ought to be irreducible, too. Hopefully, the connec- 
tion with Jordan algebras might lead to a uniform proof of the irreducibility. 

The first case is probably the most intricate. Here rk (sp 2n+2m , sp 2n ©sp 2m ) = min(n, m) = 
m. Since the commuting variety associated with (sp 2n+2 , sp 2n ©sp 2 ) is reducible, there is 
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a non-zero semisimple element h 6 0i such that is reducible. Therefore, our in- 

duction scheme does not work in this situation. Nevertheless, using ad hoc methods the 
second author has proved that is irreducible for (sp 2n+4 , sp 2n ©sp 4 ), n ^ 2. 

The problem of irreducibility of £(gi,/J also arises for second and third cases. The sec- 
ond case might be treated by induction, since all proper sub-symmetric pairs are direct 
sums of pairs of the same form and pairs (s[ 2m , sp 2m ), where the irreducibility is known. 
For the third case, all proper sub-symmetric pairs (qh, 9o,h) can be found using Proposi- 
tion ll.51 It turns out that here all commuting varieties <£(si,h) are irreducible. 

Another question is whether a nilpotent C7 -orbit in jjx gives rise to an irreducible com- 
ponent of different from <£ . Provided all £(&i,h) are irreducible, this question 
can be reduced to non-even cr-distinguished nilpotent elements. A classification of <j- 
distinguished nilpotent orbits is deduced from Djokovic's calculations |2J. According to 
Table 13 of [2], for item 3), there are two non-even cr-distinguished G -orbits in gi. Thus, 
the task of verifying irreducibility of <£(gi) in the third case seems to be manageable. 
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